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DETAILED ACTION 
Response to Amendment 

1 . In response to the office action from 4/20/2007, the applicant has submitted an 
amendment, filed 7/2/2007, arguing to traverse the art rejection based on the limitation regarding 
marking utterance sections for weighting and using the weighted utterance sections to convert a 
speaker independent model to a speaker dependent model (Amendment, Pages U-IS). 
Applicant's arguments have been ftilly considered, however the previous rejection is maintained 
due to the reasons listed below in the response to arguments. 

2. In light of the applicant's arguments with respect to Claims 1 and 8 (Amendment, Pages 
8-10), the examiner has withdrawn the previous 35 U.S.C. 101 rejection directed to non-statutory 
subject matter. 

Response to Arguments 

3. Applicant's arguments have been fully considered but they are not persuasive for the 
following reasons: 

With respect to Claims 16 and 23, the applicant argues that these claims are statutory 
because they are directed to "operations performed by a computer" (Amendment, Page 11). In 
response, the examiner notes that claims 16 and 23 remain non-statutory because they do not 
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refer to processes performed by a computer when a program on a computer readable medium is 
executed, and thus, enable the data structure's ftinctionality to be realized. Rather claims 16 and 
23 include a ^'storage medium" and the generic "a processor" (Le,, not necessarily a computer or 
in connection with further required enabling elements defined by a computer system). According 
to Warmerdam, data structures not claimed as embodied in computer readable media are 
descriptive material per se and are not statutory because they are not capable of causing 
functional change in the computer . See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 
(claim to a data structure per se held nonstatutory). Such claimed data structures do not define 
any structural and functional interrelationships between the data structure and other claimed 
aspects of the invention, which data structure's functionality to be realized. In contrast, a claimed 
computer readable permit the medium encoded with a computer program defines structural and 
functional interrelationships between the data structure and the computer software and hardware 
components which permit the data structure's functionality to be realized, and is thus statutory. 
In order to overcome this rejection, the examiner recommends changing "storage medium" to - 
computer-readable storage medium- and "processor" to -computer—. These amendments 
appear to be supported in the specification on pages 8-9. 

With respect to independent claims 1, 8, 16, and 23, the applicant argues that Nguyen et 
al (U.S, Patent: 6,272,462) fails to teach: 

a. ) an utterance, which is marked based on misrecognized sections (Amendment, Page 

12) 

b, ) weights, which are applied to the marked utterance sections (Amendment, Page 12); 

and 
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c.) a speaker dependent model, which is obtained by converting a speaker independent 
model using the weighted utterance sections (Amendment, Page 12), 

In response to argument a. (marking misrecognized sections of an utterance, Page 12), 
the examiner notes that Nguyen discloses a system wherein speech (Le„ an utterance) is 
recognized and recognition transcription results are generated (Col 3, Lines 1-46). Each n-best 
transcription, which may or may not be correct, directly maps to a specific portion of the 
utterance in the form of letters that make up a spoken word or words that make up an uttered 
sentence (CoL i. Lines 20-21 and 47-56), In the case of an incorrect transcription segment, an 
incorrect/unreliable label is generated (CoL 3, Lines 1''46). Since an incorrect/unreliable label is 
assigned to recognition transcription segment results, which directly maps to a specific utterance 
section in the form of a letter or word, the examiner maintains that Nguyen does effectively teach 
marking misrecognized utterance sections. 

In response to argument b. (applying weights to marked utterance sections), the examiner 
notes that negative weights are applied to incorrect transcription sections (CoL 3, Lines 32-56), 
which, as pointed out above, directly map to speech or utterance sections. Thus, the examiner 
notes that Nguyen does teach the weighting scheme defined in the presently claimed invention. 

In response to argument c. (conversion of a SI to an SD model using weighted utterance 
sections), the examiner notes that Nguyen teaches that the weighted transcription sections, which 
directly map to speech or utterance sections (see above), are applied to adapt a speaker 
independent model (CoL 3, Lines 47-56), Although Nguyen does not explicitly state the term 
"speaker dependent" as is noted by the applicant (''models are speaker-independent model, " 
Amendment, Page 12), Nguyen discloses that an initial model is speaker-independent and is 
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adapted with data supplied by a "new speaker". Thus the new model would be speaker 
dependent because it has been adjusted for data corresponding to a specific new speaker. 
Therefore, Nguyen discloses SI/SD model conversion as set forth in the presently claimed 
invention. 

In response to the applicant's arguments that Junqua (U,S. Patent: 6,253 J 81) fails to 
teach an average likelihood difference per frame because Junqua merely recites an average 
likelihood ratio (Amendment, Page 14), the examiner points out that although the calculation 
taught by Junqua is referred to by the temi "ratio," the calculation is actually a mean or average 
log-likelihood difference between a correct recognition segment likelihood and mean incorrect 
segment likelihood (''ratio computed as a logarithm '\ Col 5, Lines 23-45). Thus, Junqua recites 
the aforementioned claim limitation. 

In response to the applicant's argument that Junqua fails to teach "averaging the average 
likelihood over all error words" (Amendment, Page 14), the examiner points out that this 
limitation is not recited in claim 5. Instead, "averaging the average likelihood over all error 
words" is part of claim 4, which was previously indicated as containing allowable subject matter 
(Prior OA, Page 9), Thus, since Junqua was not relied upon to teach this limitation, such 
arguments are moot. 

In response to the applicant's arguments that the examiner has provided no apparent 
reason to combine the teachings of Nguyen and Junqua (Amendment, Page 15), the examiner 
notes that such motivation has been provided (see Prior OA, Pages 7-8) and is derived from the 
references themselves as is noted in the three-pronged test for obviousness (see MPEP 2105), 
Junqua teaches that his calculation used to determine whether a recognition is correct or incorrect 
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is useful in determining the confidence of a recognition result to achieve high-speed adaptation 
and improved reliability (CoL 3, Lines 29-31, Col 4, Lines 9-24, and Col 5, Lines 30-35), There 
is a reasonable expectation of success because both systems are directed to speaker adaptation 
(Prior OA, Page 7) and because Nguyen is also concerned with determining a difference 
between correct and incorrect recognitions (Col 3, Lines 1-15), Finally, with reference to the 
above arguments directed towards the teachings of Nguyen and Junqua, all of the claim 
limitations have been met. Thus, the examiner notes that because one of ordinary skill in the art 
would have been motivated to combine the teachings of Nguyen and Junqua due to the benefits 
disclosed by Junqua and the similar aim of differentiating correct/incorrect recognitions, the 
combination of Nguyen and Junqua is proper. 

The rejection of the remaining dependent claims is traversed for reasons similar to the 
independent claims (Amendment, Page 13), In regards to such arguments, see the above 
response directed towards claims 1, 8, 16, and 23. 

Claim Rejections - 35 USC § 101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

5. Claims 1-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed 
to non-statutory subject matter. 
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Claims 16 and 23 are drawn to a "instructions" per 5e, stored on a "storage medium", as 
recited in the preamble and as such represent non-statutory subject matter. See MPEP § 
2106.IV.B.l.a. 

Data structures not claimed as embodied in computer readable media are descriptive 
material per se and are not statutory because they are not capable of causing functional change in 
the computer . See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data 
structure per se held nonstatutory). Such claimed data structures do not define any structural and 
functional interrelationships between the data structure and other claimed aspects of the 
invention, which permit the data structure's functionality to be realized. In contrast, a claimed 
computer readable medium encoded with a computer program defines structural and functional 
interrelationships between the data structure and the computer software and hardware 
components which permit the data structure's functionality to be realized, and is thus statutory. 

Similarly, computer programs claimed as computer listings per se, i.e., the descriptions or 
expressions of the programs are not physical "things." They are neither computer components 
nor statutory processes, as they are not "acts" being performed. Such claimed computer 
programs do not define any structural and functional interrelationships between the computer 
program and other claimed elements of a computer , which permit the computer program's 
ftinctionality to be realized. Thus, claims 16 and 23 contain non-statutory subject matter. 
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Claim Rejections - 35 USC §102 



6. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

7, Claims 1-3, 7-11, and 15 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Nguyen et al (U,S. Patent: 6J72^462), 

With respect to Claim 1, Nguyen discloses: 

Calculating estimated weights for identified errors in recognition of utterances based on a 
reference string (calculating negative weights for misrecognitions based on an expected 
transcription sequence, Col 3, Lines 1-47); 

Marking sections of the utterances as being misrecognized and associating the estimated 
weights with the sections of the utterances (identifying correct segments and associating positive 
weights and identifying misrecognized segments and associating negative weights. Col. 3, Lines 
1-56); and 

Using the weighted sections of the utterances to convert a speaker independent model to a 
speaker dependent model (using weights for adaptation of a speaker independent model, Col. 2, 
Lines 44-55; and Col, i, Lines 48-56), 

With respect to Claim 2, Nguyen further discloses: 
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The method steps (a)-(c) are repeated at least once (iterative processing, Col. 3, Lines 43- 

60), 

With respect to Claim 3, Nguyen further discloses: 

The utterances are converted into a recognized phone string a first time through applying 
the speaker independent model and thereafter through applying the most recently obtained 
speaker dependent model (initial speaker independent model format, Col. 2, Lines 44-55; and 
adapted speaker model utilized in iterative processing, Col 4, Lines 43-60). 

With respect to Claim 7, Nguyen further discloses: 

Different misrecognized words have different weights (negative weighting utilizing a 
variable likelihood score. Col. 3, Lines 1-47). 

With respect to Claim 8, Nguyen discloses: 

Recognizing utterances through converting the utterances into a recognized string 
(recognizing speech and generating transcriptions corresponding to the input speech, Col 2, 
Line 56- Col 3, Line 56)\ 

Comparing the recognized string with a reference string to determine errors (aligning 
input speech transcriptions to expected transcriptions to determine recognition errors. Col. 3, 
Lines 1-56); 

Calculating estimated weights for identified errors in recognition of utterances 
based on a reference string (calculating negative weights for mis recognitions based on an 
expected transcription sequence, Col 3, Lines 1-47); 

Marking sections of the utterances as being misrecognized and associating the estimated 
weights with the sections of the utterances (identifying correct segments and associating positive 
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weights and identifying misrecognized segments and associating negative weights, Col. 3, Lines 
1-56); and 

Using the weighted sections of the utterances to convert a speaker independent model to a 
speaker dependent model (using weights for adaptation of a speaker independent model, CoL 2, 
Lines 44-55; and CoL i. Lines 48-56), 

With respect to Claim 9, Nguyen discloses: 

The utterances are converted into the recognized string through applying the speaker 
independent model (initial speaker independent model, Col, 2, Lines 44-55), 
With respect to Claim 10, Nguyen discloses: 

Parts (b)-(e) are repeated until differences between the reference and recognized strings 
are less than a threshold (repeating iterations until a convergence threshold is reached. Col 4, 
Lines 43-60). 

Claim 11 contains subject matter similar to Claim 3, and thus, is rejected for the same 
reasons. 

Claim 15 contains subject matter similar to Claim 7, and thus, is rejected for the same 
reasons. 

Claim Rejections - 35 USC §103 

8. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
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having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. Claims 5, 13, 16-18, 20, 22-26, 28, and 30 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Nguyen et al in view of Junqua (U,S. Patent: 6,253, 181), 

With respect to Claims 5 and 13, Nguyen discloses the method for speaker adaptation 
utilizing a weighting scheme for misrecognitions based on a likelihood score, as applied to 
Claims 1 and 8. Nguyen does not specifically disclose that calculation of a weighting score that 
computes an average likelihood difference per frame, however Junqua discloses a calculation of 
a likelihood difference used in determining a speaker adaptation that utilizes an average of 
likelihood difference scores associated with an incorrect recognition (CoL 4, Lines 9-24; and 
Col 5, Lines 15-67), Junqua further discloses an equation similar to that recited in claim 5 for 
determining a log-likelihood difference in a speaker adaptation process that utilizes an average of 
likelihood scores (Col, 5, Lines 15-67; and Col 4, Lines 9-24), 

Nguyen and Junqua are analogous art because they are from a similar field of endeavor in 
speaker adaptation systems. Thus, it would have been obvious to a person of ordinary skill in the 
art, at the time of invention, to modify the teachings of Goronzy with the likelihood difference 
calculation taught by Junqua in order to implement a high speed speaker adaptation system that 
is capable of providing a measure of recognition reliability (Junqua, Col 3, Lines 29-31; and 
Col4, Lines 9-24), 

With respect to Claims 16 and 23, Nguyen teaches the speaker adaptation system that 
utilizes weighted adaptation, as applied to claims 1 and 8. Goronzy does not specifically suggest 
method implementation as a program stored on a memory medium, however Junqua discloses a 
speaker adaptation method implemented using a processor and associated memory that provides 
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the benefit of implementing a practical speaker adaptation process in a hardware system (Col 7, 
Lines 10-14). 

Claims 17-18, 20 and 22 contain subject matter similar to claims 2-3, 5, and 7, and thus, 
are rejected for the same reasons. 

Claims 24-26, 28 and 30 contain subject matter similar to claims 9-11,13 and 15, and 

thus, are rejected for the same reasons. 

Allowable Subject Matter 

10. Claims 4, 6, 12, 14, 19, 21, 27, and 29 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

1 1 . The following is a statement of reasons for the indication of allowable subject matter: 
With respect to Claims 4, 12, 19, and 27, the prior art of record fails to explicitly teach 

or fairly suggest a method for speaker adaptation that utilizes estimated weights based on 
misrecognized speech utterances as respectively recited in claims 1,8, 16, and 23, wherein the 
estimated weights are calculated by computing an average likelihood difference per frame and 
then computing a weight value by averaging the average likelihood difference over error words 
(specification, page 6). 

Although Junqua (U.S. Patent: 6, 253 J 81) teaches an equation for calculating an average 
likelihood difference, as applied to claim 5, Junqua does not teach averaging the average 
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likelihood difference over all error words to determine a weight for speaker adaptation of a 
speech recognition model. 

With respect to Claims 6, 14, 21, and 29, the prior art of record fails to explicitly teach 
or fairly suggest a method for speaker adaptation that utilizes estimated weights based on 
misrecognized speech utterances, wherein the estimated weights are calculated by multiplying an 
average likelihood difference per frame calculated using the equation recited in claims 5, 13, 20, 
and 28 by the inverse of a number of misrecognized words for a particular speaker as per the 
equation recited in claims 6, 14, 21, and 29. 

Although Junqua (UX Patent: 6,253,181) teaches an equation for calculating an average 
likelihood difference, Junqua does not teach multiplying the calculated average likelihood by the 
inverse of a number of misrecognized words for a particular speaker as per the equation recited 
in claims 6, 14, 21, and 29. 
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Conclusion 

12. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 1 36(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

13. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: 

Zheng et al ^Efficiently Using Speaker Adaptation Data, " 2000)- disclose a method for 
converting a SI model to a SD model. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James S. Wozniak whose telephone number is (571) 272-7632. 
The examiner can normally be reached on M-Th, 7:30-5:00, F, 7:30-4, Off Alternate Fridays. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached at (^71) 272-7603. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Infomiatiori Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status infomiation for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



James S. Wozniak 
8/16/2007 



